Less well known, but just as fundamental as such visual tricks, are everyday aspects of audio perception that are "illusions" in the sense that what we perceive is very different from the reality in the physical world surrounding us. For example, if I were to say "think of a steady, continuous, unchanging sound like the tone of an organ," you could most likely do so without difficulty. In reality, however, there is nothing "steady" or "continuous" or "unchanging" about an organ sound. The physical reality is that an organ sound (like any sound) is an undulating wave with alternating regions of high and low pressure, air molecules must be constantly wiggling back and forth. Were the pressure changes and the motions to cease, then so would the sound.

Because the undulations occur at a rate too fast to perceive separately, the auditory system "blurs" them together and they achieve the illusion of steadiness. Thus sounds which occur closer together in time than the threshold of simultaneity (Fig. 5 places this at about 1 ms) are merged into a "single sound." This is parallel to the everyday illusion that television (and movies) show continuous action; in reality movies consist of a sequence of still photos which are shown at a rate faster than the threshold of simultaneity for vision, which is about 20 Hz. Closely related to the illusion of continuity is the illusion of simultaneity: sounds appear to occur at the same time instant even though in reality they do not. The ear tends to bind events together when they occur close to each other in time and to clearly separate others that may be only slightly further apart.

Another common auditory "illusion" is based on the ear's propensity to categorize certain kinds of sounds. For example, it is easy to say the vowel "a" and to slowly (and "continuously") change it into the vowel "e." Thus there is a continuum of possible vowel sounds: "a" at one end, two thirds "a" and one third "e," one half "a" and one half "e," one third "a" and two thirds "e," continuing to a pure "e" at the other end. Yet no matter who is speaking, one never perceives the intermediate vowel sounds. The sound is automatically categorized by the auditory system into either "a" or "e," never into something in between. This is called categorical perception, and it has obvious importance in the comprehension of language.

While there are many similarities between visual and auditory perceptions, there are also significant differences. For example, in 1886 Mach demonstrated that spatial symmetry is directly perceptible to the eye whereas temporal symmetry is not directly perceptible to the ear. Unlike vision, the human ability to parse musical rhythms inherently involves the measurement of time intervals.

Illusions of Pitch

Loosely speaking, pitch is the perceptual analog of frequency. Acousticians define pitch formally as "that attribute of auditory sensation in terms of which sounds may be ordered on a scale extending from low to high." Sine waves, for which the frequency is clearly defined, have unambiguous pitches because everyone orders them the same way from low to high. For non-sine wave sounds, such an ordering can be accomplished by comparing the sound with unknown pitch to sine waves of various frequencies. The pitch of the sinusoid that most closely matches the unknown sound is then said to be the pitch of that sound.

Pitch determinations are straightforward when working with musical instruments that have a clear fundamental frequency and harmonic overtones. When there is no discernible fundamental, however, the ear will often create one. Such virtual pitch occurs when the pitch of the sound is not the same as the pitch of any of its overtones. This is shown on the Auditory Demonstrations CD, where the "Westminster Chimes" song is played using only upper harmonics. In one demonstration, the sounds have spectra like that shown in Fig. 7. This particular note has partials at 780, 1040, and 1300 Hz, which is clearly not a harmonic series. These partials are, however, closely related to a harmonic series with fundamental at 260 Hz, because the lowest partial is 260 times 3, the middle partial is 260 times 4, and the highest partial is 260 times 5. The ear recreates the missing fundamental, and this perception is strong enough to support the playing of melodies, even when the particular harmonics used to generate the sound change from note to note. Thus the ear can create pitches even when there is no stimulus at the frequency of the corresponding sinusoid. This is somewhat analogous to the "triangle" that is visible in Fig. 6(d).

Perhaps the most striking demonstration that pitch is a product of the mind and not of the physical world is Shepard's ever rising tone, which is an auditory analog of the ever-ascending staircase in Fig. 6(f). Sound example EverRise presents an organ-like timbre that is constructed as diagrammed in Fig. 8. The sound ascends chromatically up the scale: after ascending one full octave, it has returned to its starting point and ascends again. The perception is that the tone rises forever (this version is about 5 minutes long) even though it never actually leaves a single octave!

Why Illusions Happen

Already we have encountered illusions of continuity and simultaneity, illusions of categorization, and two different kinds of pitch illusions (the missing fundamental and the ever-rising sound). As will become clear in Chapter 4, there are many other kinds of auditory illusions, not all of them operating at very short time scales. Why is the ear so easily fooled into reporting things that do not exist (such as the missing fundamental), into failing to sense things that do exist (such as the vowel that is in reality halfway between "a" and "e"), and into perceiving things that cannot be (such as an ever-rising pitch)?

The ear's job (to engage in some introductory chapter anthropomorphization) is to make sense of the auditory world surrounding it. This is not an easy job, because sound consists of nothing more than ephemeral pressure waves embedded in a complex three dimensional world. The sound wave that arrives at the ear is a conglomeration of sound from all surrounding events merged together. The ear must unmerge, untangle, and interpret these events. The ear must capture the essence of what is happening in the world at large, simultaneously emphasizing the significant features and removing the trivial. Does that sound represent the distant rustling of a lion or the nearby bustling of a deer? The distinction may be of some importance.

Given this formidable task, the ear has developed a sophisticated multi-level strategy. In the first stage, collections of similar sense impressions are clustered into objects of perception called auditory events. This occurs on a very short time scale. At the second stage, auditory events that are similar in some way are themselves grouped together into larger chunks, to form patterns and categories that are most likely the result of learning and experience.

For example, the low level processing might decode a complex wave into an auditory event described by the phoneme "a." This represents a huge simplification because while there are effectively an infinite variety of possible waveshapes, there are only about 45 distinct phonemes (in English). At the next stage, successive phonemes are scanned and properly chunked together into words, which can then invoke various kinds of long term memory where something corresponding to meaning might be stored.

As another example, the low level processing might decode a complex waveform into an auditory event such as the performance of a musical "note" on a familiar instrument; the C of a flute. Again, this represents a simplification because there are only a few kinds of instruments while there are an infinite variety of waveforms. At the next stage, several such notes may be clustered to form a melodic or rhythmic pattern, again, condensing the information into simple and coherent clusters that can then be presented to long term memory and parsed for meaning.

Thus the ear's strategy involves simplification and categorization. A large amount of continuously variable data arrives; a (relatively) small amount of well categorized data leaves, to be forwarded to the higher processing centers. In the normal course of events, this strategy works extremely well. If, for instance, two sounds are similar (by beginning at the same time, by having a common envelope, by being modulated in a common way, by having a common period, by arriving from the same direction, etc.) then they are likely to be clustered into a single event. This makes sense because in the real world, having such similarities implies that they are likely to have arisen from the same source. This is the ear doing its job.

If we, as devious scientists, happen to separate out the cues associated with legitimate clustering and to manipulate them independently, then it should come as no surprise that we can "fool" the ear into perceiving "illusions." The pitch illusions are of exactly this kind. It would be a rare sound in the real world that would have multiple harmonically related partials yet have no energy at the frequency corresponding to their common period (such as occurs in Fig. 7). It would be an even rarer sound that spanned the complete audio range in such a way that the highest partials faded out of awareness exactly as the lowest partials entered.